{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "yeadDkMiISin"
      },
      "source": [
        "# Gemini API: Safety Quickstart"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "lEXQ3OwKIa-O"
      },
      "source": [
        "<table align=\"left\">\n",
        "  <td>\n",
        "    <a target=\"_blank\" href=\"https://colab.research.google.com/github/google-gemini/cookbook/blob/main/quickstarts/Safety.ipynb\"><img src=\"https://www.tensorflow.org/images/colab_logo_32px.png\" />Run in Google Colab</a>\n",
        "  </td>\n",
        "</table>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "uOxMUKTxR-_j"
      },
      "source": [
        "The Gemini API has adjustable safety settings. This notebook walks you through how to use them. You'll write a prompt that's blocked, see the reason why, and then adjust the filters to unblock it.\n",
        "\n",
        "Safety is an important topic, and you can learn more with the links at the end of this notebook. Here, we're focused on the code."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "9OEoeosRTv-5"
      },
      "outputs": [],
      "source": [
        "!pip install -q -U google-generativeai # Install the Python SDK"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "TS9l5igubpHO"
      },
      "outputs": [],
      "source": [
        "import google.generativeai as genai"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "gHYFrFPjSGNq"
      },
      "source": [
        "## Set up your API key\n",
        "\n",
        "To run the following cell, your API key must be stored it in a Colab Secret named `GOOGLE_API_KEY`. If you don't already have an API key, or you're not sure how to create a Colab Secret, see the [Authentication](https://github.com/google-gemini/cookbook/blob/main/quickstarts/Authentication.ipynb) quickstart for an example."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "ab9ASynfcIZn"
      },
      "outputs": [],
      "source": [
        "from google.colab import userdata\n",
        "GOOGLE_API_KEY=userdata.get('GOOGLE_API_KEY')\n",
        "genai.configure(api_key=GOOGLE_API_KEY)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "LZfoK3I3hu6V"
      },
      "source": [
        "## Prompt Feedback\n",
        "\n",
        "The result returned by the [Model.generate_content](https://ai.google.dev/api/python/google/generativeai/GenerativeModel#generate_content) method is a [genai.GenerateContentResponse](https://ai.google.dev/api/python/google/generativeai/types/GenerateContentResponse)."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "2bcfnGEviwTI"
      },
      "outputs": [],
      "source": [
        "model = genai.GenerativeModel('gemini-1.0-pro')\n",
        "\n",
        "unsafe_prompt =   # Put your unsafe prompt here\n",
        "response = model.generate_content(unsafe_prompt)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "WR_2A_sxk8sK"
      },
      "source": [
        "This response object gives you safety feedback in two ways:\n",
        "\n",
        "* The `prompt_feedback.safety_ratings` attribute contains a list of safety ratings for the input prompt.\n",
        "\n",
        "* If your prompt is blocked, `prompt_feedback.block_reason` field will explain why."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "8887de812dc0"
      },
      "outputs": [],
      "source": [
        "bool(response.prompt_feedback.block_reason)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "he-OfzBbhACQ"
      },
      "outputs": [],
      "source": [
        "response.prompt_feedback.safety_ratings"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "72b4a8808bb9"
      },
      "source": [
        "If the prompt is blocked because of the safety ratings, you will not get any candidates in the response:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "f20d9269325d"
      },
      "outputs": [],
      "source": [
        "response.candidates"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "4672af98ac57"
      },
      "source": [
        "### Safety settings"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "2a6229f6d3a1"
      },
      "source": [
        "Adjust the safety settings and the prompt is no longer blocked. The Gemini API has four configurable safety settings."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "338fb9a6af78"
      },
      "outputs": [],
      "source": [
        "response = model.generate_content(\n",
        "    unsafe_prompt,\n",
        "    safety_settings={\n",
        "        'HATE': 'BLOCK_NONE',\n",
        "        'HARASSMENT': 'BLOCK_NONE',\n",
        "        'SEXUAL' : 'BLOCK_NONE',\n",
        "        'DANGEROUS' : 'BLOCK_NONE'\n",
        "    })"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "86c560e0a641"
      },
      "source": [
        "With the new settings, the `blocked_reason` is no longer set."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "0c2847c49262"
      },
      "outputs": [],
      "source": [
        "bool(response.prompt_feedback.block_reason)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "47298a4eef40"
      },
      "source": [
        "And a candidate response is returned."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "028febe8df68"
      },
      "outputs": [],
      "source": [
        "len(response.candidates)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ujVlQoC43N3B"
      },
      "source": [
        "You can check `response.text` for the response."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "de8ee74634af"
      },
      "outputs": [],
      "source": [
        "response.text"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "3d401c247957"
      },
      "source": [
        "### Candidate ratings"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "3d306960dffb"
      },
      "source": [
        "For a prompt that is not blocked, the response object contains a list of `candidate` objects (just 1 for now). Each candidate includes a `finish_reason`:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "e49b53f69a2c"
      },
      "outputs": [],
      "source": [
        "candidate = response.candidates[0]\n",
        "candidate.finish_reason"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "badddf10089b"
      },
      "source": [
        "`FinishReason.STOP` means that the model finished its output normally.\n",
        "\n",
        "`FinishReason.SAFETY` means the candidate's `safety_ratings` exceeded the request's `safety_settings` threshold."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "2b60d9f96af0"
      },
      "outputs": [],
      "source": [
        "candidate.safety_ratings"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "n1UdbxVt3ysY"
      },
      "source": [
        "## Learning more\n",
        "\n",
        "Learn more with these articles on [safety guidance](https://ai.google.dev/docs/safety_guidance) and [safety settings](https://ai.google.dev/docs/safety_setting_gemini).\n",
        "\n",
        "## Useful API references\n",
        "\n",
        "There are 4 configurable safety settings for the Gemini API:\n",
        "* `HARM_CATEGORY_DANGEROUS`\n",
        "*`HARM_CATEGORY_HARASSMENT`\n",
        "* `HARM_CATEGORY_SEXUALLY_EXPLICIT`\n",
        "* `HARM_CATEGORY_DANGEROUS`\n",
        "\n",
        "Note: while the API [reference](https://ai.google.dev/api/python/google/ai/generativelanguage/HarmCategory) includes others, the remainder are for older models.\n",
        "\n",
        "* You can refer to the safety settings using either their full name, or the aliases like `DANGEROUS` used in the Python code above.\n",
        "\n",
        "Safety settings can be set in the [genai.GenerativeModel](https://ai.google.dev/api/python/google/generativeai/GenerativeModel) constructor.\n",
        "\n",
        "* They can also be passed on each request to [GenerativeModel.generate_content](https://ai.google.dev/api/python/google/generativeai/GenerativeModel#generate_content) or [ChatSession.send_message](https://ai.google.dev/api/python/google/generativeai/ChatSession?hl=en#send_message).\n",
        "\n",
        "- The [genai.GenerateContentResponse](https://ai.google.dev/api/python/google/ai/generativelanguage/GenerateContentResponse) returns [SafetyRatings](https://ai.google.dev/api/python/google/ai/generativelanguage/SafetyRating) for the prompt in the [GenerateContentResponse.prompt_feedback](https://ai.google.dev/api/python/google/ai/generativelanguage/GenerateContentResponse/PromptFeedback), and for each [Candidate](https://ai.google.dev/api/python/google/ai/generativelanguage/Candidate) in the `safety_ratings` attribute.\n",
        "\n",
        "- A [glm.SafetySetting](https://ai.google.dev/api/python/google/ai/generativelanguage/SafetySetting)  contains: [glm.HarmCategory](https://ai.google.dev/api/python/google/ai/generativelanguage/HarmCategory) and a [glm.HarmBlockThreshold](https://ai.google.dev/api/python/google/generativeai/types/HarmBlockThreshold)\n",
        "\n",
        "- A [glm.SafetyRating](https://ai.google.dev/api/python/google/ai/generativelanguage/SafetyRating) contains a [HarmCategory](https://ai.google.dev/api/python/google/ai/generativelanguage/HarmCategory) and a [HarmProbability](https://ai.google.dev/api/python/google/generativeai/types/HarmProbability)\n",
        "\n",
        "The [glm.HarmCategory](https://ai.google.dev/api/python/google/ai/generativelanguage/HarmCategory) enum includes both the categories for PaLM and Gemini models.\n",
        "\n",
        "- When specifying enum values the SDK will accept the enum values themselves, or their integer or string representations.\n",
        "\n",
        "- The SDK will also accept abbreviated string representations: `[\"HARM_CATEGORY_DANGEROUS_CONTENT\", \"DANGEROUS_CONTENT\", \"DANGEROUS\"]` are all valid. Strings are case insensitive."
      ]
    }
  ],
  "metadata": {
    "colab": {
      "toc_visible": true,
      "provenance": []
    },
    "google": {
      "image_path": "/static/site-assets/images/docs/logo-python.svg",
      "keywords": [
        "examples",
        "gemini",
        "beginner",
        "googleai",
        "quickstart",
        "python",
        "text",
        "chat",
        "vision",
        "embed"
      ]
    },
    "kernelspec": {
      "display_name": "Python 3",
      "name": "python3"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}